Training Feed-Forward Neural Networks with Monotonicity Requirements

In this paper, we adapt the classical learning algorithm for feed-forward neural networks when monotonicity is require in the input-output mapping. Such requirements arise, for instance, when prior knowledge of the process being observed is available. Monotonicity can be imposed by the addition of suitable penalization terms to the error function. The objective function, however, depends nonlinearly on the first-order derivatives of the network mapping. We show that these derivatives can easily be obtained by an extension of the standard back-propagation algorithm procedure. This yields a computationally efficient algorithm with little overhead compared to back-propagation.